Review Findings

P1 - Adjacent trace tests fail after the invalid-score error changed

apps/cli/src/commands/inspect/utils.ts:137 now routes standalone JSONL loading through normalizeResultRow(), so rows with a missing or non-numeric score throw Unsupported result row ... Expected an AgentV result row... instead of the prior Missing or invalid score... message. The existing trace utility contract at apps/cli/test/commands/trace/trace.test.ts:191 and apps/cli/test/commands/trace/trace.test.ts:198 still asserts the old message, and bun test apps/cli/test/commands/trace/trace.test.ts now fails those two tests. CI runs the full test suite, so this should block merge until the trace tests are updated to the new migration guidance or loadJsonlRecords() preserves the trace-specific invalid-score error.

Verification

• bun install and bun run build were needed first because this fresh worktree lacked workspace package dist output.
• bun test apps/cli/test/commands/results passed: 193 pass.
• bun test apps/cli/test/commands/eval/artifact-writer.test.ts passed: 45 pass.
• bun run typecheck passed.
• bun apps/cli/src/cli.ts results summary apps/cli/test/fixtures/results/camel-replay returned failed_test_ids: ["wtg-replay-fail"], total_duration_ms: 1234, and total_tokens: 15.
• bun apps/cli/src/cli.ts inspect filter apps/cli/test/fixtures/results/camel-replay/index.jsonl --has-tool rg --format json returned the replay row with tool_names: ["rg"].
• Adjacent check bun test apps/cli/test/commands/trace/trace.test.ts failed: 47 pass, 2 fail, both invalid-score message expectations.

Note: gh pr review --request-changes was rejected by GitHub because the authenticated user cannot request changes on their own pull request, so I am leaving this as a blocking review comment.

Deploying agentv with    Cloudflare Pages

View logs

Addressed the blocking trace-test comment in c234fd04.

Decision: preserved the trace-specific invalid-score contract for standalone trace/result JSONL loaded through loadResultFile(). That path accepts trace exports, so Missing or invalid score... is more precise there than the result-artifact migration guidance. Eval-case-only rows still go through the shared result-row schema error and keep the migration guidance.

Validation:

• Reproduced before fix: bun test apps/cli/test/commands/trace/trace.test.ts failed 2 invalid-score expectations.
• bun test apps/cli/test/commands/trace/trace.test.ts — 49 pass.
• bun test apps/cli/test/commands/results — 193 pass.
• bun test apps/cli/test/commands/eval/artifact-writer.test.ts — 45 pass.
• bun run typecheck — pass.
• bun run lint — pass.

CI Test job follow-up after c234fd0 / failed job https://github.com/EntityProcess/agentv/actions/runs/27683502887/job/81876531953

Root cause from the log:

• apps/cli/test/commands/compare/compare.test.ts still expected the result/trace-specific Missing or invalid score contract, but invalid result-shaped rows were being rejected by the shared migration-guidance schema error before compare's own validation ran.
• apps/cli/test/commands/results/shared.test.ts built the camel replay fixture path from process.cwd(), which becomes apps/cli under the CLI package test job and produced apps/cli/apps/cli/....

Fix in 239ab033:

• normalizeResultRow now keeps eval-case-only rows on the migration-guidance error, while result-shaped rows with missing/non-finite score keep the existing invalid-score schema error.
• Nested trace/output normalization no longer writes undefined fields onto rows, so eval-case rows do not get misclassified as result-like.
• The camel replay fixture path is resolved relative to shared.test.ts, not cwd.

Validation:

• bun test apps/cli/test/commands/compare/compare.test.ts apps/cli/test/commands/results/shared.test.ts — 55 pass
• bun test apps/cli/test/commands/results — 193 pass
• bun test apps/cli/test/commands/eval/artifact-writer.test.ts — 45 pass
• bun test apps/cli/test/commands/trace/trace.test.ts — 49 pass
• bun test apps/cli/test/commands/compare/compare.test.ts — 50 pass
• bun run typecheck — pass
• bun run lint — pass
• git diff --check — pass

Re-review result

No blockers remain for the prior results-schema feedback.

Verified the requested points:

• Trace invalid-score expectations are fixed: bun test apps/cli/test/commands/trace/trace.test.ts passed, 49 tests.
• Historical camelCase replay rows still normalize: results/shared.test.ts and artifact-writer.test.ts cover the fixture path, timing/token/cost/status aliases, and trace.toolCalls; an extra smoke confirmed canonical test_id/duration_ms values win when both snake_case and camelCase are present.
• Compare/results fixture lookup no longer depends on process.cwd(): root-run tests passed, and bun test test/commands/results/shared.test.ts test/commands/compare/compare.test.ts also passed from apps/cli cwd, 55 tests.
• Canonical snake_case result rows remain preferred, while eval-case-only JSONL still rejects with the migration guidance (Eval-case JSONL is input data...).

Local verification:

• bun install and bun run build for this fresh worktree setup.
• bun test apps/cli/test/commands/trace/trace.test.ts — 49 pass.
• bun test apps/cli/test/commands/results — 193 pass.
• bun test apps/cli/test/commands/compare/compare.test.ts — 50 pass.
• bun test apps/cli/test/commands/eval/artifact-writer.test.ts — 45 pass.
• bun test test/commands/results/shared.test.ts test/commands/compare/compare.test.ts from apps/cli — 55 pass.
• bun run typecheck — pass.
• git diff --check origin/main...HEAD — pass.

GitHub checks on the latest commit are green: Build, Test, Typecheck, Lint, Validate Evals, Validate Marketplace, Check Links, and Cloudflare Pages all pass.

Residual risk: this was a targeted re-review of the prior blockers rather than a full functional UAT pass. Given the local targeted coverage plus green CI, I do not see remaining blocking risk for PR #1400.